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Abstract. The use of online translation (OT) is increasing as more pupils receive 
laptops from their schools. This study investigates OT use in two groups of 
Swedish pupils (ages 17-18) studying Spanish as an L3: one group (A) having 
free Internet access and the spelling and grammar checker of Microsoft Word, the 
other group (B) using printed dictionaries without Internet access. 112 Spanish 
essays were collected from the groups. Screencasts of 60 essay writings were 
recorded, and accompanied by a questionnaire and interviews to 13 pupils. 
The essays were analysed for morphological, syntactical and lexical accuracy. 
Significant differences between groups A and B were found for only a few error 
categories. Group B made a higher percentage of errors all-in-all, committed more 
mistakes regarding noun/adjective and noun/article agreement, whereas group A 
made more mistakes concerning verb mood, personal pronouns, and conjunctions. 
Many errors in group B can be explained by the fact that the pupils had no access 
to automatic corrective suggestions or automatic translation, as did group A. 
Flaws in OT can account for pronoun and syntactic errors in group A essays. The 
differences in correct use of verb mood and conjunctions are more difficult to 
explain and deserve further investigation. 
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1. Introduction 

As pupils’ access to computers is increasing at schools investing in 1-1 projects 
in Sweden as well as other countries, their access to online resources is also 
increasing. This is evident in foreign language classes when it comes to writing 
in the studied language. Teachers are often heard complaining about pupils using 
online translation (OT) services, such as Google translate; however, the number 
of studies on pupils’ use of OT in foreign language writing, especially concerning 
younger pupils, is still scarce, with exceptions from recent years such as Nino (2008, 
2009), Steding (2009), Garcia and Pena (2011), and O’Neill (2012). Teachers’ and 
pupils’ ideas of OT are often based more on personal beliefs and prejudices, rather 
than on detailed data. The present study, part of a Ph.D. project on pupils’ use and 
perception of ICT in foreign language learning, examines the effects of OT on 
grammatical, syntactic, lexical and pragmatic features of Spanish essays written by 
pupils at the Swedish upper secondary school. 

2. Methods and participants 

2.1. Participants and data collection 

Two groups studying Spanish as a foreign language were investigated during the 
autumn semester of 20 1 3 as they wrote four essays on text genres present in the 
Swedish national curriculum for foreign languages (Skolverket, 2013). The pupils, 
ages 17-18, had studied Spanish for five years prior to the study. Both groups 
used their laptops to write and both had access to printed dictionaries. One group 
(henceforth “A”) had free access to the Internet and to the spelling and grammar 
checkers of Microsoft Word, whereas the other group (“B”) was not allowed to 
use any online resources. Group A mainly chose to use Google translate and the 
similar site Lexikon24; a few Google searches for background information on 
different topics were also made. In all, 57 pupils handed in 112 essays, 84 from 
group A and 28 from group B. The fact that several group B pupils had not 
followed instructions during the writing sessions, using Internet resources despite 
being told not to, disqualified many of their essays from the analysis. 

Each of the four writing tasks were distributed via the school’s learning 
management system, where the pupils also handed in their essays. They were given 
thirty minutes to write, with an additional five minutes to read instructions and to 
save their work. They were also asked to record their screens as they were writing, 
using screencast-o-matic.com. Thirty-three pupils managed to do so, handing in 
screencasts of 60 essay writings. 
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After the final essay writing session, the pupils were also asked to fill out a 
questionnaire, and thirteen pupils were interviewed. The questions asked focussed 
on the pupils’ choice of writing strategies and technology use and will not be 
further commented on here. 

2.2. Measures of morphosyntactic accuracy 

Morphological or grammatical accuracy was measured as the mean number of 
morphological errors per essay, regarding verbs, noun/adjective and noun/article 
agreement, pronouns, and prepositions. Errors of word order, clause and sentence 
structuring, the use of adjectives instead of adverbs or vice-versa, and the use of 
conjunctions were counted to measure syntactic accuracy. 

2.3. Measures of lexical-pragmatic accuracy 

The lexical accuracy was measured as the ratio of context inadequate words to 
the total amount of words, and the amount of misspelled words (following the 
definitions of Rimrott & Heift, 2005, p. 21). 

Pragmatic accuracy regarded the choice of context appropriate words, idiomatic 
expressions, and the inconsistent use of personal pronouns and corresponding verb 
endings, with an incoherent alternation of 2nd and 3rd person as form of address, 
present in many of the group A essays. 

The data received were entered in SPSS and submitted to a t-test. The significant 
(and in a few cases near-significant) differences between the groups thus revealed 
will be commented on in the following section. 

3. Results 

The most striking result of the present study is the low number of significant 
differences between the groups, as far as morphology and syntax are concerned. 
Bigger differences, mainly regarding complexity, could be found on clause and 
sentence levels, where group B produced fewer sentences, fewer independent 
clauses and a smaller amount of correct sentences; the differences are, however, 
also quite small in these fields, albeit significant. 

On the whole, group B made more errors (p=. 012), but many of these are smaller 
mistakes such as noun/adjective or noun/article agreement, and misspellings. 
Taking into account the pupils’ Spanish grades from the previous year, the 


98 


Effects of online translation on morpho syntactic and lexical-pragmatic accuracy... 


differences both within and between the groups are clearly correlated to grade 
level, high-achievers from both groups performing better, producing fewer 
errors, longer texts and more complex sentence structures. 

3.1. Morphosyntactic accuracy 

As for morphology, significant differences were found concerning the use of verb 
mood, where group A performed more errors (p=.007), as well as noun/adjective 
agreement (near significant, p=. 052) and noun/article agreement (p=.001), both 
showing more errors in group B. 

Group A produced more syntactic inaccuracies than group B. These inaccuracies 
regard minor errors as the appropriate choice of conjunctions (a near significant 
difference, />=.057), and more important syntactic errors affecting entire 
clauses or sentences, often rendering them difficult or impossible to understand 
Cp=.036). 


Significant differences were also found in the ratio of correct sentences to the total 
number of sentences, group A rating higher (p=,013). The accuracy on sentence 
level refers to both morphological and syntactic accuracy as well as the absence 
of misspellings, in numbers, the differences are quite small, group A on average 
producing 11.90 sentences per essay out of which 3 correct, and group B 10.57 
sentences, out of which 2 correct. The differences are more clearly referable to 
grade levels than to technology use. 

3.2. Lexical and pragmatic accuracy 

Group B made many more spelling errors than group A, which is natural as they 
did not have any help other than dictionaries, and, furthermore, often chose not to 
consult these as it was considered too time-consuming or too difficult to find the 
words or to understand the dictionary entries. 

Pupils of group A commented that they felt that the Spanish spelling and grammar 
checker of Microsoft Word (that was not used in group B) had helped them to 
correct many mistakes. This seems plausible as far as misspellings are concerned, 
but less so for grammatical errors, which were still numerous. Many of the spelling 
errors were automatically corrected by Word (especially missing accents and 
erroneous use of capital letters), and these corrections do not seem to have attracted 
pupils’ attention; therefore, their belief that the spelling checker helped them to 
learn Spanish orthography better must be taken with a pinch of salt. 
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In the first essay, written as a reply to a fictitious 19-year-old boy asking for advice 
on his friends’ overconsumption of alcohol, group A switched frequently between 
formal and informal forms of address (using the Spanish tu and listed and their 
corresponding 2nd and 3rd person verb endings). This was seen to a much lesser 
extent in group B. The majority of these inconsistencies were clearly due to the use 
of OT, as Google translate often -but not always- translated the Swedish du (2nd 
person) to Spanish listed (3rd person). 

4. Discussion and conclusions 

Apart from the few areas where significant differences could be seen, the use of OT 
does not seem to have affected writing performance in any decisive way, neither 
improving it nor giving worse residts than had it not been used. Group A essays 
were slightly longer and contained a few more complex sentences and fewer errors, 
but almost none of the 112 essays can be considered well-written. Only one of the 
57 pupils managed to use the technology effectively, producing texts in a Spanish 
appropriate for the level of instruction. 

If the purpose of intermediate level foreign language writing is considered strictly 
as text production and nothing more, the present study can neither recommend 
nor discourage the use of OT, as its effects seem hardly discernable. The role of 
writing in the foreign language classroom needs, however, to be taken into greater 
consideration; most teachers would hopefully say that its purpose is to practice and 
improve pupils’ abilities to use the language, and in that case, OT should probably 
be used sparingly, as suggested by researchers stating that it does not improve 
language learning (Garcia & Pena, 2011; Nino, 2008, 2009; O’Neill, 2012; 
Steding, 2009). This is, nevertheless, an area that needs further research rather than 
a priori sentiments of rejection. Considering the difficulties among the pupils’ of 
the present study to produce grammatically and pragmatically coherent texts or, 
indeed, sentences, the Swedish school needs to reconsider first and foremost the 
place foreign language writing should have in foreign language teaching, and only 
secondly what role OT and other technologies should play in this. 
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